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TITLE OF THE INVENTION 

CORRECTION OF BARYCENTERS OF SIGNALS OBTAINED BY 
ADDING AND READING CHARGES ACCUMULATED IN SOLID- 
STATE IMAGE SENSING DEVICE 

FIELD OF THE INVENTION 

The present invention relates to a signal 
processing apparatus, an image sensing apparatus 
such as a video camera, and a control method of a 
signal processing apparatus and more specifically 
to, a signal processing apparatus, an image 
sensing apparatus, and a control method of a 
signal processing apparatus which processes an 
image signal outputted from a solid-state image 
sensing device selectively controllable in a mode 
of reading signals separately from respective 
pixels of an image sensing area, and a mode of 
adding and reading the signals by plurality of 
pixels • 

BACKGROUND OF THE INVENTION 

Conventionally, for an image sensing 
apparatus which combines and reads signal charges 
generated by respective photoelectric conversion 
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elements, a method of driving image sensing 
elements so as to combine signal charges 
corresponding to the same color component in 
vertical transfer registers is considered (refer 
to Japanese Patent Application Laid-Open No. 2000- 
253415) . 

Figs . 4A and 4B show a conceptual view of 
combining charges for the same colors in the 
vertical direction accumulated in image sensing 
elements covered with a Bayer filter. The Bayer 
filter has R, G, and B color filters arranged as 
shown in Fig. 4A. More specifically, lines in 
which R and G filters are alternately arranged, 
and lines in which G and B filters are alternately 
arranged are disposed alternately. When signal 
charges are combined in the vertical direction, 
since filters of each color are disposed on every 
other pixels, a driving unit of the image sensing 
elements drives the image sensing elements so that 
signal charges of the pixels arranged in every 
other lines are combined. 

For example, referring to Fig. 4A, pixel 
signals from a line of R 00 and G 01 are combined with 
pixel signals from a line of R 20 and G 21 , then pixel 
signals of a line of R 10 and G 1X shown in Fig. 4B 
are generated. Further, pixel signals from a line 
of G 10 and B X1 are combined with pixel signals from 
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a line of G 30 and B 31 , then pixel signals of a line 
of G 20 and B 21 are generated. Similarly, pixel 
signals from a line of R 40 and G 41 are combined with 
pixel signals from a line of R 60 and G 61 , then pixel 
5 signals of a line of R 50 and G 51 are generated, and 
pixel signals from a line of G 50 and B 51 are 
combined with pixel signals from a line of G 70 and 
B 71 ,then pixel signals of a line of G 60 and B 61 are 
generated. The color order of the combined signals 

10 is the same as the Bayer order of the signals 
before combined. 

When signals are combined in the aforesaid 
manner, the barycenter of each combined signal is 
located in the center of the two pixels before 

15 combined. This makes the barycenters of the 

combined signals approach each other by two lines 
as shown in Fig. 4B. Namely, the intervals between 
the barycenters are unequal in the vertical 
direction. Therefore, it is necessary to apply 

20 camera signal processing conforming to such 

arrangement to the combined signals. Whereas, when 
signal charges are separately read from the image 
sensing elements without being combined for, e.g., 
sensing a still image, camera signal processing 

25 conforming to the normal Bayer arrangement has to 
be performed. Thus, a camera signal processing 
unit conforming to a case of reading charge 
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signals without combining them and a camera signal 
processing unit conforming to a case of combining 
and reading charge signals for sensing a moving 
image are needed. 

SUMMARY OF THE INVENTION 

The present invention has been made in 
consideration of the above situation, and has as 
its object to enable to perform the same camera 
signal processing both in a case of reading charge 
signals without combining them and in a case of 
combining and reading charge signals. 

According to the present invention, the 
foregoing object is attained by providing a signal 
processing apparatus for processing an image 
signal outputted from an image sensing device 
which has a plurality of photoelectric conversion 
elements covered with a color filter and which can 
be driven in a first reading method of separately 
reading signals from the respective photoelectric 
conversion elements and a second reading method of 
adding signals generated by the photoelectric 
conversion elements by at least two signals 
corresponding to the photoelectric conversion 
elements of a same color then outputting the added 
signals, comprising: a switch that switches 
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between the first reading method and the second 
reading method; and a correction unit that passes 
signals inputted from the image sensing device 
without correcting positions of barycenters of the 
5 inputted signals when the first reading method is 
set, and corrects positions of barycenters of the 
inputted signals when the second reading method is 
set • 

According to the present invention, the 

10 foregoing object is also attained by providing an 
image sensing apparatus comprising: an image 
sensing device; a driving unit that drives the 
image sensing device; and the image processing 
apparatus having the configuration as described in 

15 the proceeding paragraph • 

According to the present invention, the 
foregoing object is also attained by providing a 
signal processing method for processing an image 
signal outputted from an image sensing device 

20 which has a plurality of photoelectric conversion 
elements covered with a color filter and which can 
be driven in a first reading method of separately 
reading signals from the respective photoelectric 
conversion elements and a second reading method of 

25 adding signals generated by the photoelectric 
conversion elements by at least two signals 
corresponding to the photoelectric conversion 
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elements of a same color then outputting the added 
signals, comprising: determining which of the 
first reading method and the second reading method 
is set; and correcting positions of barycenters of 
5 the signals inputted from the image sensing device 
when the second reading method is set. 

In a preferred embodiment, in the correcting, 
the positions of barycenters of the inputted 
signals are corrected so that the intervals 
10 between the barycenters become equal in the 

vertical direction when the second reading method 
is set. 

Other features and advantages of the present 
invention will be apparent from the following 
15 description taken in conjunction with the 

accompanying drawings, in which like reference 
characters designate the same or similar parts 
throughout the figures thereof. 

20 BRIEF DESCRIPTION OF THE DRAWINGS 

The accompanying drawings, which are 
incorporated in and constitute a part of the 
specification, illustrate an embodiment of the 
25 invention and, together with the description, 

serve to explain the principles of the invention. 

Fig. 1 is a block diagram illustrating a 
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brief configuration of an image sensing apparatus 
according to an embodiment of the present 
invention ; 

Figs. 2A and 2B are explanatory views for 
explaining a correction operation for correcting 
deviation of barycenters according to the 
embodiment of the present invention; 

Fig. 3 is a flowchart of control processing 
according to the embodiment of the present 
invention; and 

Figs . 4A and 4B are conceptual views of 
combining pixel charges according to the prior art. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 

A preferred embodiment of the present 
invention will be described in detail in 
accordance with the accompanying drawings . 

Fig. 1 is a block diagram of an image 
sensing apparatus according to the embodiment of 
the present invention. In Fig. 1, reference 
numeral 2 denotes an image sensing device, of. 
Inter alia, CCD or MOS type, for converting light 
flux which incidents on the image sensing device 2 
into charge signals in correspondence with the 
amount of light; 4, an A/D converter for 
converting an analog signal outputted from the 
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Image sensing device 2 into a digital signal; 6, a 
barycenter correction unit for applying barycenter 
deviation correction (will be described later) to 
digital signals outputted from the A/D converter; 
5 8, a camera signal processor for applying, inter 
alia, color separation, aperture process, y 
correction to the digital signal outputted from 
the barycenter correction unit 6 and generating a 
luminance signal and color difference signals; 10, 

10 a driving unit for generating a timing signal for 
controlling the image sensing device 2; and 12, an 
image sensing mode setting unit for changing 
between a non-addition mode (still image sensing 
mode) of separately reading signal charges 

15 acquired by the image sensing device 2 without 
adding the signal charges, and an addition mode 
(moving image sensing mode) of reading signal 
charges from the image sensing device 2 after 
combining the signal charges of the same color. 

20 When the non-addition mode (still image 

sensing mode) is set by the image sensing mode 
setting unit 12, the driving unit 10 controls the 
image sensing device 2 so that signal charges are 
sequentially outputted without being combined, and 

25 when the addition mode is set, signal charges are 
read in such a manner as described in the 
background of the invention with reference to Figs. 
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4A and 4B. 

Further, referring to Fig. 3, when the non- 
addition mode is set by the image sensing mode 
setting unit 12 ("No" in step S10), the barycenter 
5 correction unit 6 does not perform the barycenter 
deviation correction and simply passes the digital 
signal inputted from the A/D converter 4 to the 
camera signal processor 8 without processing it 
(step S12). Whereas, if the addition mode is set 
10 ("Yes" in step S10), the barycenter correction 
unit 6 performs the barycenter deviation 
correction (step Sll), and outputs the corrected 
signals to the camera signal processor 8 (step 
S12). 

15 Therefore, in the non-addition mode, when 

the image sensing device 2 is covered with a Bayer 
filter as shown in Fig. 4A, the color order of 
read signal charges corresponds to the color 
arrangement of the filter, and the barycenter 

20 correction unit 6 simply passes the signals in the 
read order to the camera signal processor 8. 
Therefore, the camera signal processor 8 applies 
signal processes conforming to the image sensing 
device with the Bayer filter to the signals 

25 outputted from the barycenter correction unit 6, 
and outputs a luminance signal and color 
difference signals . 
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Whereas, in the addition mode (moving image 
sensing mode), signals corresponding to the same 
color are combined and read as described in the 
background of the invention with reference to Figs. 
5 4A and 4B. 

The color arrangement of the combined 
signals is similar to the Bayer arrangement of the 
signals before combined. However, as described in 
the background of the invention, a barycenter of 

10 each added signal is located in the center of the 
two added pixel, and the intervals between the 
centers of the added signals are unequal in the 
vertical direction, i.e., lines of the added 
pixels approaches by two lines as shown in Fig. 4B. 

15 Accordingly, the barycenter correction unit 

6 processes the combined signals so that intervals 
between the barycenters of the combined signals 
become equal when the addition mode is selected by 
the image sensing mode setting unit 12. 

20 Figs. 2A and 2B are explanatory views for 

explaining barycenter deviation correction 
according to the embodiment. Fig. 2A shows 
combined signals of the same color pixels 
outputted from the A/D converter 4 when the signal 

25 addition is performed as described with reference 
to Figs. 4A and 4B. Here, let P 2n and P 2n -2 denote 
even number lines, and the even number lines be 
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the upper lines of the approached two lines due to 
the barycenter deviation. Further, let P 2n+1 and 
P 2n-i denote odd number lines, and the odd number 
lines be the lower lines of the approached two 
5 lines due to the barycenter deviation. 

Further, let P ' 2n and P' 2 n-2 denote even number 
lines after the barycenter deviation correction, 
and let P'^.j and P 1 2n .n denote odd number lines 
after the barycenter deviation correction. n is a 

10 natural number. 

As shown in Fig. 2A, since the ratios of the 
interval between the P 2n _ 2 and P 2n . 1 and the interval 
between the P 2nll and P 2n are 1:3, the barycenter 
correction unit 6 of this embodiment performs 

15 correction using the following equations so as to 
make the interval between the lines equal as shown 
in Fig. 2B: 

P' 2n = l/8Xp 2n _ 2 + 7/8Xp 2n 
P' 2 n.i = 7/8XP 2n . 1+ l/8Xp 2n+1 

20 As for the signals in the even number lines, 

signals in a given line (e.g., signals of P 2n line) 
and signals in a line two lines above the given 
line (e.g., signals of P 2n _ 2 line) are added with 
the weight of 7:1. Further, as for the signals in 

25 the odd number lines, signals in a given line 

(e.g., signals of P 2n _! line) and signals in a line 
two lines below the given line (e.g., signals of 
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P 2n+1 line) are weighted averaged with the weight 
ratio of 7:1. 

In the color order of the signals after 
being combined as shown in Fig. 2A, since the 
5 signals of the same color are arranged every other 
pixel, the interpolated (corrected) signals 
expressed by the above equations are calculated 
using pairs of combined signals in every other 
lines . 

10 Fig. 2B shows a concept of the signals 

obtained by the barycenter deviation correction as 
described above. 

After the barycenter correction unit 6 
performs the aforesaid processing, corrected 

15 combined signals whose barycenters now separate at 
a uniform interval in the vertical direction as 
shown in Fig. 2B are inputted to the camera signal 
processor 8 shown in Fig. 1. The combined pixel 
signals which have undergone the barycenter 

20 deviation correction can be treated equally to 
signal charges whose color order corresponds to 
the normal Bayer arrangement. Therefore, it is 
possible to obtain a luminance signal and color 
difference signals by applying the same camera 

25 signal processes performed in the non-addition 
mode to the corrected combined signals. 

It should be noted that the barycenter 
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deviation is corrected using 

P' 2n = l/8Xp 2n _ 2 + 7/8Xp 2n 

PSn-i - 7/8Xp 2n . 1+ l/ 8 Xp 2n+1 
However, the method of correcting the barycenter 
5 deviation is not limited to this, and it is 

possible to modify the present invention to use 
other barycenter deviation correction method. 

Further, in the aforesaid explanation, a 
case of using a Bayer color filter of the primary 
10 colors is described. However, the present 
invention is not limited to this, and is 
applicable to a color filter of any color 
arrangement as far as, when the charges are read 
out in the addition mode, intervals between the 
15 barycenters of the combined signals are unequal in 
the vertical direction. 

Further, a case of adding charges in two 
lines is explained; however, the present invention 
may be also applied to a case of adding charges in 
20 three or more lines. 

< Other Embodiment > 

The present invention can be applied to a 
system constituted by a plurality of devices (e.g., 
host computer, interface, camera head) or to an 
25 apparatus comprising a single device (e.g., 
digital still camera, digital video camera) . 

Further, the object of the present invention 
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can also be achieved by providing a storage medium 
storing program codes for performing the aforesaid 
processes to a computer system or apparatus (e.g., 
a personal computer), reading the program codes, 
by a CPU or MPU of the computer system or 
apparatus, from the storage medium, then executing 
the program. 

In this case, the program codes read from 
the storage medium realize the functions according 
to the embodiment, and the storage medium storing 
the program codes constitutes the invention. 

Further, the storage medium, such as a 
floppy disk, a hard disk, an optical disk, a 
magneto-optical disk, CD-ROM, CD-R, a magnetic 
tape, a non-volatile type memory card, and ROM, 
and computer network, such as LAN (local area 
network) and WAN (wide area network) , can be used 
for providing the program codes. 

Furthermore, besides aforesaid functions 
according to the above embodiment are realized by 
executing the program codes which are read by a 
computer, the present invention includes a case 
where an OS (operating system) or the like working 
on the computer performs a part or entire 
processes in accordance with designations of the 
program codes and realizes functions according to 
the above embodiment. 
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Furthermore, the present invention also 
includes a case where, after the program codes 
read from the storage medium are written in a 
function expansion card which is inserted into the 
computer or in a memory provided in a function 
expansion unit which is connected to the computer, 
CPU or the like contained in the function 
expansion card or unit performs a part or entire 
process in accordance with designations of the 
program codes and realizes functions of the above 
embodiment . 

In a case where the present invention is 
applied to the aforesaid storage medium, the 
storage medium stores the program codes 
corresponding to the flowchart of Fig. 3 and a 
correction operation corresponding to Figs. 2A and 
2B described in the embodiment . 

The present invention is not limited to the 
above embodiments and various changes and 
modifications can be made within the spirit and 
scope of the present invention. Therefore to 
apprise the public of the scope of the present 
invention, the following claims are made. 



